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1.0  INTRODUCTION 

An  important  step  in  the  intelligence  gathering  process  is  the  fusion  of  information  provided  by  several 
sources.  The  objective  of  this  process  is  to  build  an  up-to-date  and  correct  view  of  the  current  situation 
with  the  overall  available  information  in  order  to  make  adequate  decisions.  Moreover,  to  succeed  in  this 
process,  it  is  important  to  associate  with  each  available  information,  some  attributes  like  the  number  of  the 
sources  that  support  it,  their  reliability,  the  degree  of  truth  of  the  information  etc.  For  the  moment,  these 
attributes  are  managed  by  the  human  operator  when  he  fuses  information  provided  by  the  different 
sources.  However,  there  is  no  real  methodology  to  do  this  in  a  formal  manner.  And,  when  relaying 
this  fusion  process  to  a  machine,  we  need  to  develop  formal  definitions  and  algorithms  to  manage 
these  attributes  in  addition  to  fusing  information.  Furthermore,  in  a  context  of  interoperability  where 
different  systems  exchange  information,  common  definitions  of  these  attributes  have  to  be  shared. 
The  Standardization  Agreements  (STANAG)  2022  of  North  Atlantic  Treaty  Organization  (NATO)  defines 
a  framework  for  such  common  definitions. 

The  purpose  of  this  paper  is  first  to  analyze  the  STANAG  2022  recommendations  about  information 
evaluation  and  then  to  set  a  first  step  in  the  definition  of  a  formal  and  non  ambiguous  system  for 
evaluating  information.  Indeed,  as  it  will  be  shown,  the  present  recommendations,  written  in  natural 
language  are  rather  ambiguous  and  imprecise  and  are  open  to  discussion.  This  paper  is  organized  as 
follows.  Section  2  presents  a  review  of  STANAG  2022  recommendations  and  points  out  to  the  main 
notions  that  underline  these  recommendations.  Section  3  analyzes  the  different  assumption  underlying  the 
evaluation  proposed  in  the  STANAG.  Section  4,  an  example  of  symbolical  formalism  for  the  fusion  of 
information  is  presented.  In  this  example,  we  consider  some  of  the  assumption  of  STANAG  2002.  Finally, 
section  5  is  devoted  to  a  discussion. 


2.0  REVIEW  OF  STANAG  2022  RECOMMENDATIONS 

The  Annex  to  STANAG  2022,  Edition  8  ([1])  explicitly  mentions  that  the  aim  of  information  evaluation  is 
to  indicate  the  degree  of  confidence  that  may  be  placed  in  any  item  of  information  which  has  been 
obtained  for  intelligence.  (...)  This  is  achieved  by  adopting  an  alphanumeric  system  of  rating  which 
combines  a  measurement  of  the  reliability  of  the  source  of  information  with  a  measurement  of  the 
credibility  of  that  information  when  examined  in  the  light  of  existing  knowledge. 

Examining  the  whole  text  leads  us  to  point  out  that  the  two  main  concepts  in  this  evaluation  system  are  the 
reliability  of  the  sources  and  the  credibility  of  the  information.  These  concepts  are  defined  in  the 
STANAG  2022  recommendations,  as  follows: 
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Reliability  of  the  source  is  designated  by  a  letter  between  A  and  F  signifying  various  degrees  of 
confidence  as  indicated  below. 

•  a  source  is  evaluated  A  if  it  is  completely  reliable.  It  refers  to  a  tried  and  trusted  source  which  can 
be  depended  upon  with  confidence. 

•  a  source  is  evaluated  B  if  it  is  usually  reliable.  It  refers  to  a  source  which  has  been  successfully 
used  in  the  past  but  for  which  there  is  still  some  element  of  doubt  in  particular  cases. 

•  a  source  is  evaluated  C  if  it  is  fairly  reliable.  It  refers  to  a  source  which  has  occasionally  been  used 
in  the  past  and  upon  which  some  degree  of  confidence  can  be  based. 

•  a  source  is  evaluated  D  if  it  is  not  usually  reliable.  It  refers  to  a  source  which  has  been  used  in  the 
past  but  has  proved  more  often  than  not  unreliable. 

•  a  source  is  evaluated  E  if  it  is  unreliable.  It  refers  to  a  source  which  has  been  used  in  the  past  and 
has  proved  unworthy  of  any  confidence. 

•  a  source  is  evaluated  F  if  its  reliability  cannot  be  judged.  It  refers  to  a  source  which  has  not  been 
used  in  the  past. 

Credibility  of  information  is  designated  by  a  number  between  1  and  6  signifying  varying  degrees  of 
confidence  as  indicated  below. 

•  If  it  can  be  stated  with  certainty  that  the  reported  information  originates  from  another  source  than 
the  already  existing  information  on  the  same  subject,  then  it  is  classified  as  "confirmed  by  other 
sources"  and  rated  1 . 

•  If  the  independence  of  the  source  of  any  item  of  information  cannot  be  guaranteed,  but  if,  from 
the  quantity  and  quality  of  previous  reports,  its  likelihood  is  nevertheless  regarded  as  sufficiently 
established,  then  the  information  should  be  classified  as  ''probably  true"  and  given  a  rating  of  2. 

•  If,  despite  there  being  insufficient  confimation  to  establish  any  higher  degree  of  likelihood, 
a  freshyly  reported  item  of  information  does  not  conflict  with  the  previously  reported  behaviour 
pattern  of  the  target,  the  item  may  be  classified  as  "possibly  true"  and  given  a  rating  of  3. 

•  An  item  of  information  which  tends  to  conflict  with  the  previously  reported  or  established 
behaviour  pattern  of  an  intelligence  target  should  be  classified  as  "doubtful"  and  given  a  rating 
of  4. 

•  An  item  of  information  which  positively  contradicts  previously  reported  information  or  conflicts 
with  the  established  behaviour  pattern  of  an  intelligence  target  in  a  marked  degree  should  be 
classified  as  "improbable"  and  given  a  rating  of  5. 

•  An  item  of  information  is  given  a  rating  of  6  if  its  truth  cannot  be  judged. 

The  previous  definitions  of  information  evaluation  can  be  criticized.  Indeed,  since  they  are  given  in  natural 
language,  they  are  quite  imprecise  and  ambiguous  and  a  lot  of  points  are  open  to  discussion.  For  exemple, 
if  we  consider  the  credibility  of  information,  it  seems  that  the  rating  defined  according  to  the 
recommendations  does  not  describe  a  unique  property.  For  instance,  how  should  we  qualify  an  item  of 
information  supported  by  several  sources  of  information  which  is  also  conflictual  with  some  already 
registered  information?  According  to  STANAG  definitions,  this  item  should  be  given  a  rating  of  1  and 
should  also  be  given  a  rating  of  5. 

Furthermore,  according  to  the  recommendations,  a  rating  of  6  should  be  given  to  an  item  whose  truth 
cannot  be  judged.  This  suppposes  that  the  other  ratings  (1...5)  concern  the  evaluation  of  the  truth  of 
information.  If  so,  the  rating  of  1  should  be  given  to  a  true  information.  But,  as  it  is  defined,  the  rating  of 
1  is  given  to  an  item  supported  by  at  least  two  sources.  This  is  questionable  since,  the  different  sources 
(even  if  they  agree)  may  emit  false  information. 
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However,  even  if  the  previous  recommendations  can  be  criticized,  we  can  see  that  they  present  three  basic 
concepts  that  present  a  cornerstone  in  evaluation  system.  These  concept  are  the  following: 

•  the  reliability  of  a  source 

•  the  number  of  independent  sources  that  support  an  information 

•  the  fact  that  the  information  tends  to  conflict  with  some  available  information. 

It  is  clear  that  these  three  concepts  are  independent,  and  that  they  have  to  be  included  in  all  quotation 
systems,  whether  these  systems  are  automatic  or  not. 


3.0  ANALYSIS  OF  THE  DIFFERENT  EVALUATION  CONCEPTS 

As  it  was  noticed  above  three  basic  concepts  are  underlying  the  operational  notion  of  quotation. 

Nevertheless,  these  concepts  are  used,  in  the  STANAG,  in  an  operational  framework  which  is  not 
necessarily  usable  in  an  automatic  process.  For  this  reason,  we  try  in  this  section  to  give  a  more 
comprehensive  and  formal  definitions  of  these  concepts  with  the  aim  of  using  them  in  an  automatic 
process  of  fusion. 

•  Reliability 

In  mathematical  logic,  the  reliability  of  a  source  is  defined  in  a  binary  way  as  follows:  an  information 
source  is  (totally)  reliable  if  and  only  if  the  information  it  delivers  are  true  in  the  real  world. 
For  instance,  a  sensor  which  measures  the  temperature  is  totally  reliable  if  and  only  if  the  temperature  it 
indicates  is  the  correct  one;  a  human  expert  is  totally  reliable  if  and  only  if  any  information  (opinion, 
conjecture  etc)  he  gives  is  true. 

According  to  the  recommendations  of  STANAG  2022,  ,  the  reliability  of  a  source  is  not  a  binary  notion 
but  a  graded  one  and  is  defined  in  reference  to  its  use  in  the  past.  It  can  be  measured  for  example,  as  the 
ratio  of  the  number  of  times  the  source  gave  a  true  information  to  the  number  of  times  it  gave 
information.  However,  this  definition  does  not  take  into  account  the  actual  environment  of  use  of  the 
information  source.  For  instance,  even  if  it  is  known  to  be  reliable,  an  infra-red  sensor  loose  reliability 
when  it  rains.  So  we  have  to  take  into  account  the  condition  in  which  the  source  is  used.  Some  practical 
consideration  of  the  use  of  this  notion  in  a  numerical  context  can  be  find  in  [7],  [8]. 

However,  it  must  be  noticed  that  there  is  no  consensus  yet  on  a  formal  definition  of  reliability. 
For  instance,  we  can  read  in  the  APJ  2.1  “ every  piece  of  information  produced  by  an  impeccable  source  is 
not  necessarily  correct”.  If  “impeccable”  intends  to  mean  “reliable”,  this  sentence  is  contradictory  with 
the  definition  given  previously.  Here,  it  implies  that  the  reliability  of  a  source  is  not  defined  by  its  ability 
to  deliver  truth  and  that  even  a  reliable  source  can  be  wrong.  But  it  can  be  wrong  not  because  it  is  not 
sincere  but  because  its  model  of  discernment  is  maybe  not  precise  enough  to  distinguish  true  from  false. 

•  Independence 

This  notion  have  to  be  a  little  bit  enlightened.  The  classical  statistical  definition  of  the  independence  of 
two  events  is  given  by  the  rule: 

two  events  A  and  B  are  independent  iff  P(A,B)=P(A)P(B) 

The  STANAG  highlights  the  independence  of  sources  to  confirm  information.  Indeed  this  independence  is 
important  if  we  want  to  improve  our  confidence  in  a  decision  when  many  sources  agree  on  the  same 
decision.  However  this  condition  on  independence  is  not  sufficient.  Let  us  consider  n  sources  given  a 
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decision  about  an  hypothesis  H;.  Our  confidence  that  this  hypothesis  is  true  given  the  fact  that  the 
n  sources  agree  on  the  fact  that  this  hypothesis  is  true  is  given  by  the  probability:  P(H,/d|=i,  ...,dn=i) 
this  probability  may  be  written  in  terms  of  elementary  probability  of  each  source  by  the  formula: 


P(Hj  /  dj  =  i,...,  dn  =  i)  = 


P(d,  =  =  i/Hi)P(Hi) 

P(dj  =i,...,dn  =  i) 


For  the  sake  of  simplicity  if  we  suppose  that  elementary  sources  are  independent,  then  the  above  formula 
may  be  written  as: 


]^[  P(dj  =  i/Hj)P(Hj) 

P(H,/d,  =  i,...,dr  =  i)  =  — — = - 

■  1  n  p(di=o 


Furthermore  if  we  suppose  that  each  source  has  the  same  performance,  it  is  easy  to  see  that  this  probability 
is  higher  than  the  probability  that  the  hypothesis  is  true  given  only  one  source  if  and  only  if  each  of  the 
source  is  informative: 


P(dj=i/Hi)AP(dj=i) 

This  means  that  the  probability  of  giving  a  right  decision  for  a  given  hypothesis  is  higher  than  the 
probability  of  giving  the  same  decision  no  matter  what  is  the  hypothesis. 

•  Conflict 

In  this  part  we  address  the  concept  of  conflict  with  a  focus  on  it  use  in  a  symbolical. 

In  mathematical  logic,  conflict  is  defined  by  the  notion  of  contradiction.  Some  pieces  of  information,  i.e, 
some  formulas,  are  conflictual  if  and  only  if,  they  are  contradictory  (i.e,  there  exists  no  possible  world  in 
which  they  are  all  true). For  instance,  “the  object  is  a  plane”,  “the  object  is  an  helicopter”  are  two 
conflictual  pieces  of  information  (given  the  background  knowledge  that  no  object  can  be  both  a  plane  and 
an  helicopter),  “its  speed  is  600kmh”  “its  speed  is  625kmh”  are  also  are  two  conflictual  pieces  of 
information  (since  obviously,  not  both  of  them  can  be  true). 

However,  it  seems  that  the  STANAG  recommendations  want  to  make  a  distinction  between  information 
which  are  conflictual  and  information  which  tend  to  be  in  conflict. 

The  notion  of  conflict  may  be  related  to  the  notion  of  distance  between  information.  In  logic  an 
information  is  modelised  by  a  propositional  variable.  Then  the  conflict  between  tow  information  may  be 
modelised  by  a  distance  which  is  null  if  the  propositional  formulas  are  not  in  conflict  and  different  of  zero 
if  they  are. 

In  the  following  we  give  some  definition  and  property  of  the  notion  of  distance  in  the  context  of  logic  for 
dealing  with  the  credibility  given  in  the  STANAG. 

definition 

Let  L  a  propositional  language  on  a  finite  alphabet  of  propositional  variables  <P.  An  interpretation  is  a 
mapping  from  <P  toward  {0,1}.  The  set  of  all  the  interpretation  is  write  W.  An  interpretation  I  is  a  model  of 
a  formula  if  and  only  if  this  formula  is  true  for  this  inteipretation.  Let  w  define  a  formula  and  mod(w).is 
the  set  of  model  of  w.  That  is  mod(cp)={Ie\V  / 1  1=  cp}. 
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Let  us  now  define  a  distance  between  two  interpretations.  This  is  a  function  d  :  WxW  — »N  such  that: 
d(I,J)  =  d(J,I) 
d(I,J)  =  0  if  I=J 

This  notion  helps  us  to  naturally  define  a  distance  between  an  interpretation  and  a  propositional  variable 
by: 


d(I,  cp)  =  min  d(I,J) 

J  |=q> 

Now  it  is  possible  to  extend  this  definition  to  the  distance  between  an  interpretation  and  a  set  of 
propositional  variables.  For  this,  we  have  to  introduce  an  operator  for  the  combinatory  of  the  elementary 
distances.  The  choice  of  this  operator  is  outside  the  scope  of  this  paper  and  we  only  focus  here  on  two 
operator:  sum  and  max.  With  these  operators  the  distances  are  given  by: 


n 

ds(I,v|f)  =  ^d(I,(p;)  for  the  sum  operator 

i=l 

dM(I,v|/)  =  Max  d(I,cpi )  for  the  max  operator 

i=l,...,n 


With  Y|/  ={(pi,  ...,cpn} 

In  the  symbolical  formalism,  the  simplest  distance  that  it  is  possible  to  define  between  two  interpretations 
is  the  drastic  distance.  This  distance  gives  0  if  the  two  inteipretations  are  equal  and  1  if  not. 


dD(U) 


JO  si  I  =  J 
[  1  sinon 


In  this  case  the  distance  between  an  interpretation  and  a  set  of  logical  proposition  is  also  equal  to  0  or  1 . 

Given  w  and  w’  two  models  of  a  formula,  the  number  of  propositional  letter  whose  valuation  differs  from 
w  to  w’  is  called  the  hamming  distance. 


4.0  EXAMPLE  FOR  A  FORMAL  MODEL  OF  EVALUATION  FOR 
SYMBOLICAL  INFORMATION 

In  this  paragraph,  we  give  some  ideas  on  how  an  evaluation  as  close  as  possible  to  the  one  recommended 
by  the  STANAG  may  be  use  when  dealing  with  symbolical  information  The  fusion  process  we  illustrate 
on  an  example  takes  into  account  the  number  of  the  sources  that  support  an  information,  their  reliability. 
However  it  does  not  take  into  account  a  graded  notion  of  conflict. 

•  Classical  fusion  operator 

We  recall  here  some  definitions  introduced  by  Konieczny  and  Pino-Perez  [2],  [3]  in  the  context  of  fusion 
of  database. 
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Let  dbl...dbn  be  n  sets  of  information  and  Poss  be  a  set  of  models.  Konieczny  and  Pino-Perez  defined  a 
majority  merging  operator,  denoted  A  such  that  the  set  of  models  of  the  information  source  which  is 
obtained  from  merging  dbl...  dbn  with  this  operator,  Mod(ifs(\db,,...,dbn  ]))),  is  semantically 
characterized  by: 

Mod(A([dbl,...,dbn ]))  =  min  jPoss) 


where  <[dbl . .  .dbn]  is  a  total  pre-order  on  Poss  defined  by  : 

w<dK..dK  V  iff  d(w, [dbj.-.db n ]) <  d(w’ , [db , . . .db n ]) 

with 

n 

d  (w,  [db, . .  .dbn  1)  =  X  w,  mn  ^ }  d  (w,  w') 

where  d(w,w')  is  the  Hamming  distance.  In  other  words,  when  merging  dbl. ..dbn  with  the  operator  A,  the 
result  is  semantically  characterized  by  the  models  in  Poss  which  are  minimal  according  to  the  pre-order 

—  \db\,...,dbn\  ' 

•  Fusion  operator  with  reliability  of  the  sources 

Here,  we  to  extend  this  operator  by  taking  a  weighted  sum  instead  of  the  sum. 

We  assume  now  that  any  db;  is  associated  with  a  weight,  i.e,  an  integer  denoted  r(db,)  representing  its 
degree  of  reliability.  For  instance,  a  source  whose  reliability  is  A  (resp,  B,  C,  D)  will  be  weighted  by 
5  (resp,  4,  3,  2,  1). 

We  define  Aws  by: 


Mod  (A  ws([dbl,...,db  ,,]))=  min  (Pass) 


-[db,  db]  is  a  t°tal  pre-order  on  Poss  defined  by: 

ta<E...^„]  w'  iff  dws(w,[db,...db„])<  dws(w' ,[db,...dbn]) 

with 


d ws (w, [db,.. .dbn  1)  =  X  h „  min ^ ( d (w,  w').r(dbi ) 


i.e 


d ws  (w,  [db, . .  .dbn  ])  =  £  r(dbt ).  h„  min /;  d  (w,  w') 


i= 1 
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•  Example  1 

We  consider  that  the  problem  consists  in  identifying  a  flying  object.  This  object  can  be  a  plane  (p)  or  an 
helicopter  (h)  or  a  missile  (m).  Assume  we  have  five  information  sources  called  One,  Two,  Three,  Four 
and  Five  whose  reliability  are  respectively  A,  B,  C,  D  and  A.  We  consider  the  following  flow  of 
information: 

•  One  emits  p  v  h 

•  Two  emits  h  v  m 

•  Three  emits  p 

•  Four  emits  h 

•  Five  emits  p. 

The  questions  that  raise  are:  is  the  object  a  plane?  is  the  object  an  helicopter  ?  is  the  object  a  missile  ? 

By  the  previous  definitions,  we  can  define: 

db,  ={pvh}  and  r(db, )  =  5 
db2  ={hvm}  and  r(db2 )  =  4 
db3  =  {p}  and  r(db3)=  3 
db4  =  {h}  and  r(db4)=  2 
db 5  =  {p}  and  r(db5)=  5 

The  three  possible  worlds  are: 

w,  =  — m}  in  which  the  object  is  a  plane, 

w2  =  { — i /? ,  h,  — i m)  in  which  the  object  is  an  helicopter, 
vv3  =  {—ip,—ih,m}  in  which  the  object  is  a  missile 


Then  we  can  compute  the  following  distances: 

>12 


dws 

,w19 

db1 

..db5 

dws 

,W2> 

db! 

...db5 

dws 

,W3> 

db, 

...db5 

then  Mod((A([dblv..,db5])))  =  w,  =  {p,  — ih,  — m} 


It  means  that,  given  the  information  transmitted  until  now  by  the  different  sources  and  given  their 
reliabilities,  the  most  plausible  answer  to  the  query  “what  is  the  object?”  is  “the  object  is  a  plane”. 

•  Example  2 

Now,  let  us  consider  that  the  previous  information  sources  have  different  degrees  of  reliability, 
for  instance: 


r(db,  )  =  2 
r(db2)=  5 
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r(db3)=  2 
r(db4)=  5 
r(db5)=  2 

And  suppose  that  the  five  sources  send  the  same  information  as  before. 
Then  the  distances  are  now  the  following: 


dWS 

,wl> 

db, 

..db5 

dWS 

,W2> 

dbj 

...db5 

dws 

!W3> 

db3 

...db5 

which  give  as  a  result:  “it  is  an  helicopter”. 

This  example  shows  the  impact  of  the  reliability  of  the  sources  in  the  result  of  the  decision:  changing 
reliability  degrees  of  the  sources  may  lead  to  different  results. 


5.0  DISCUSSION 

This  work  intended  to  formalize  some  informal  recommendations  about  information  evaluation  in 
information  fusion.  Some  of  the  informal  notions  underlying  the  recommendations  have  been  given  a 
formal  interpretation,  even  if,  as  it  has  been  shown,  no  consensus  exist  yet  on  these  definitions  and  more 
work  is  needed.  Based  on  this  formalization,  we  have  suggested  to  implement  the  fusion  process  by  a 
weighted  majority  because  this  fusion  operator  takes  into  account  the  number  of  the  sources  and  their 
reliability.  If  the  number  of  the  sources  had  not  been  so  important,  we  could  have  chosen  a  method  taking 
into  account  the  reliability  of  the  sources  only  (see  [4],  [5],  [6]  for  instance). 

However,  some  of  the  notions  underlying  the  STANAG  recommendations  have  not  yet  been  taken  into 
account. 

For  instance,  in  classical  literature,  we  made  no  difference  between  "being  conflictual"  and  "tending  to  be 
confhctual".  This  last  notion  assumes  that  there  is  a  scale  for  defining  conflicts.  Generally,  this  scale  is 
binary  and  the  formal  notion  which  represents  the  conflict  is  the  logical  inconsistency.  Extension  to  graded 
conflict  may  be  modeled  thank  to  the  definition  of  distance  between  interpretation  but  further  work  have 
to  be  made  to  go  closer  to  the  STANAG  recommendation. 

Another  point  which  has  been  left  aside  is  the  total  ignorance  about  the  reliability  of  a  given  source,  which 
was  a  case  foreseen  in  the  recommendations.  In  the  present  formalization,  it  seems  difficult  to  represent 
that.  Indeed,  which  number  can  be  attached  to  a  source  whose  reliability  is  not  known?  That  is  an  open 
question. 
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France 


Problem 


Why  fusion  ? 


To  obtain  a  reliable  et  accurate  information  in  all  conditions  of 
observation 

The  problem  is  that  all  information  collected  is  not  of  a  perfect 
accuracy  or  reliability 


O  N  E  R  A 


Objective  of  data  fusion 


Suppose  all  sources  are  perfectly  accurate  and  reliable 


O  N  E  R  A 


Objective  of  data  fusion 


To  obtain  a  reliable  et  accurate  information  in  all  conditions 
of  observation 


It  is  a  mig 

ft  ig  pm  FI  % 

1 4 

It  si  a  plane 

Need  for  a 
FUSION 


Difficulties 


The  first  difficulty  Is  that  the  frame  of  discernment  of  the  source  may  be  very 
different 

-  MTI  Radar :  wheeled  vehicle,  rotating  blades,  armored  vehicle 

-  ESM  :  PR4G,  Radar,  ... 

-  Humint : . 

The  second  difficulty  is  that  there  is  no  general  framework  to  deal  with  the 
evaluation  of  the  uncertainty  of  an  information 


o  N  E  R  A 


Objective  here 


Our  purpose  here  address  only  the  issue  of  improving  the 

reliability  of  information  through  fusion.  That  is,  we  want  a  true 
information,  one  which  is  a  good  representation  of  the  reality. 

The  problem  of  accuracy  is  related  to  that  ofTaxonomie/Ontologie 


O  N  E  R  A 


Existing  frameworks 


Usually  two  different  frameworks  exist  depending  on  the  nature  of  the  data  but 
above  all  on  the  mathematical  culture  of  the  researcher  who  is  dealing  with 

Numerical  data:  generally  provided  by  sensor 

The  techniques  to  deal  with  are:  probability,  possibility 
theory,  Dempster-Shafer  theory 

Symbolical  data:  generally  provided  by  Human 

The  techniques  to  deal  with  is  :  logic,  ... 


o  N  E  R  A 


The  road  map  for  automatic  evaluation 


Stanag  2022  give  some  road  map  to  evaluate  data  whatever  the  provider 
(sensor  or  humint)  but  need  some  human  expertise  to  deal  with. 

This  stanag  is  well  adapted  for  human  treatment  of  information  but  needs 
some  mathematical  formalism  to  be  included  in  an  automatic  treatment 

It  distinguishes  two  types  of  evaluation: 

-  the  reliability  of  a  source 

-  the  credibility  of  an  information 


o  N  E  R  A 


Evaluation  of  information 


11 


A  is  always  wrong 
B  is  confident 

Result :  it  is  an  F 1 8 

o  N  E  R  A 


B 


Operational  quotation 
Stanag  2022,  AJP  2.1 
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Reliability  of  the  Source 

A :  Completely  reliable 

B  :  usual  reliable 

C  :  fairly  reliable 

D  :  unusual  reliable 

E  :  unreliable 

F  :  reliability  cannot  be 
judged 


Credibility  of  the  information 
1 :  Confirmed 

2  :  probably  true 

3  :  possibly  true 

4  :  doubtful 

5  :  improbable 

6:  credibility  cannot  be 
judged 
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Reliability 
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Reliability  of  the  sources  :  In  order  to  qualify  the  source  reliability,  STANAG  2022 
recommends  to  use  letters  A...F,  so  that : 

a  source  is  evaluated  A  if  it  is  completely  reliable  (it  refers  to  a  tried  and  trusted 
source  which  can  be  depended  upon  with  confidence) 

a  source  is  evaluated  B  if  it  is  usually  reliable  (it  refers  to  a  source  which  has 
been  successfully  used  in  the  past  but  for  which  there  is  still  some  element  of 
doubt  in  particular  case) 

a  source  is  evaluated  C  if  it  is  fairly  reliable  (it  refers  to  a  source  which  has 
occasionally  been  used  in  the  past  and  upon  which  some  degree  of 
confidence  can  be  based) 

a  source  is  evaluated  D  if  it  is  not  usually  reliable  (it  refers  to  a  source  which  has 
been  used  in  the  past  but  has  proved  more  often  than  not  unreliable) 

a  source  is  evaluated  E  if  it  is  unreliable  (it  refers  to  a  source  which  has  been 
used  in  the  past  and  has  proved  unworthy  of  any  confidence) 

a  source  is  evaluated  F  if  its  reliability  cannot  be  judged  (it  refers  to  a  source 
which  has  not  been  used  in  the  past) 
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Credibility  of  the  information  :  In  order  to  qualify  the  information  credibility, 
STANAG  2022  recommends  to  use  numbers  1  ...6  so  that : 

a  piece  of  information  is  evaluated  1  (Confirmed)  if  it  is  confirmed  by 
others 

a  piece  of  information  is  evaluated  2  (probably  true)  if  its  likelihood  is 
regarded  as  sufficiently  established 

a  piece  of  information  is  evaluated  3  (possibly  true)  if  it  does  not  conflict 
with  previous  information 

a  piece  of  information  is  evaluated  4  (doubtful)  if  it  tends  to  conflict  some 
previous  information 

a  piece  of  information  is  evaluated  5  (improbable)  if  it  positively 
contradicts  previous  information 

a  piece  of  information  is  evaluated  6  if  its  truth  cannot  be  judged 


O  N  E  R  A 
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Model 


RF 


Three  different  notions 


Reliability  of  a  source 
Credibity  of  information 


-  Independence  and  the  number  of  the  sources 


Conflict  of  an  information 


Reliability  of  a  source 
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It  is  dependent  on  of  the  ability  of  a  source  to  give  a  true  information  in  a  particular 
context  of  uses 


Context 


no  jamming  daylight  no  jamming  night 

jamming  daylight 

F 

r  y 

c 

f  y 

\ 

0 

1  ' 

3  < 

r 

Sensors 


CONTEXTUAL  SPACE 


C2°C3 
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Independence 
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The  sources  have  to  be  independent  an  the  object  observed  have  also  to 
be  unique  (physical  object) 


There  is  one 
platoon 


There  is  one 
platoon 


They  saw  our 
platoon 


They  saw  our 
platoon  C=1 


They  saw  our 
platoon 
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Credibility  of  information 
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We  have  to  define  the  notion  of  conflict.  This  notion  is  closely  related  to  the 
notion  of  the  distance  a  source  may  have  between  the  data  it  received  and 
the  model  it  has. 

-  Numerical:  it  is  given  by  the  probability/possibility  of  the  data  received 
given  each  hypothesis 

P(x/H1),  P(x/H2), . ,P(x/Hn) 

-  Symbolical:  it  is  given  by  the  distance  between  interpretation 

Example  :  the  Hamming  distance  between  two  interpretations  w  and  w’  of  a  formula  is  the 
number  of  propositional  letters  which  are  true  in  w  and  false  in  w’. 


O  N  E  R  A 


A  mathematical  model  for  credibility  in  relation  with  the 

stanag  2022 
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Closed  Word  assumption:  all  the  hypotheses  are  known  by  the  user. 

In  that  case,  without  any  information  but  a  priori  information  all  hypotheses  are 
possible  so  their  credibility  is  equal  to  3  :  possible 

If  one  information  is  given  by  a  source  that  confirms  one  of  the  hypotheses 
then  this  hypothesis  has  a  credibility  of  2  :  probable  (it  is  not  confirmed)  and 
the  other  has  the  credibility  of  4  (doubtful) 

If  two  or  more  information  are  given  by  different  and  independent  sources  that 
confirm  one  hypothesis  then  the  credibility  of  this  hypothesis  is  1 
(confirmed)  and  the  credibility  of  the  other  is  5  (improbable) 
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A  mathematical  model  for  credibility  in  relation  with  the 
stanag  2022  (possibility  theory) 

Thank  to  the  possibility  theory  and  the  relation  n(  H;  )=  1  -  N(">Hj) 


n(  Hi)  =1  N(Hi)=1  then  the  credibility  is  equal  to  1  “confirmed” 
n(  Hi)  =1  0<N(Hi)<1  then  the  credibility  is  equal  to  2  “probable” 
n(  Hi)  =1  N(Hi)=0  then  the  credibility  is  equal  to  3  “possible” 
0<n(  Hi)<1  N(Hi)=0  then  the  credibility  is  equal  to  4  “doubtful” 
n(  Hi)  =0  N(Hi)=0  then  the  credibility  is  equal  to  5  “improbable” 
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Use  of  evaluation  in  a  fusion  process  (numerical) 

System  N  =  {1,2,3} 


Association  of  sensors  J  =  {1 ,2} 


Inclusive  probability 

P(C,  n  Cj )  =  J  Min(p,  (y),  n,  (y))P(y)dy 

Exclusive  probability 

pj=p(cj)=  X(-^I_J|pcnCi) 

{IcN/JcI}  ld 


O  N  E  R  A 


STATIC  ESTIMATION 


Minimum  Mean  Square  Error  Estimation 

x  =E(x/Y)  =  JxP(x/Y)dx 

Total  Probability  Theorem 

P(x/Y)=  XP(x/Y?Cj)P(Cj) 

JcN 


x=  ZPjE(x/Yj) 

JczN 


Estimation 


11 


Use  of  evaluation  in  a  fusion  process  (symbolical) 


Let  mod((p)={leW  / 1  |=  cp}  be  the  set  of  models  with  I  an  interpretation 

Given  a  distance  d  between  interpretation  :  WxW  ->N  such  that: 

d(l,J)  =  d(J,l) 
d(l,J)  =  0  if  l=J 

A  distance  between  an  proposition  and  an  interpretation  is  given  by 

d(I,cp)  =  min  d(I,J) 

J  |=cp 


O  N  E  R  A 
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Use  of  evaluation  in  a  fusion  process  (symbolical) 


The  distance  between  an  interpretation  and  a  set  of  propositions  is  given  by 


dwsM9i-<P„])=£  min  d(w, w').r((Pi ) 

1=1  w  eMod((p  ) 
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Example 

p=  plane,  h=  helicopter ,  m=missile 

•  Source  One  emits  pvh 

•  Source  Two  emits  h  v  m 

•  Source  Three  emits  p 

•  Source  Four  emits  h 

•  Source  Five  emits  p 

The  questions  are: 

•  is  the  object  a  plane  ? 

•  is  the  object  an  helicopter  ? 

•  is  the  object  a  missile  ? 
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Example 


<Pi={pvh}  r(cpj )  =  5 
cp2  =  {hvm}  r(cp2)=  4 

<P3={P}  fftH 
cp4  =  {h}  r(<p4)=2 

ip5={p}  r((Ps)=5 
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Example 


Merge(DB)  =  w,  =  {p,— ig,— m} 
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Conclusion 
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We  have  placed  emphasis  on  the  evaluation  process  on  data  fusion. 

-  We  have  to  define  two  quantities  :  the  credibility  of  information  and  the  reliability  of  the 
source 

-  This  mathematical  definition  of  these  quantities  must  have  a  real  correspondence  with  the 
heuristic  but  operational  definition  given  in  stanag  2022 

-  We  have  given  an  example  of  how  using  such  an  evaluation  in  a  fusion  process  with 
symbolical  data.  Other  works  exist  for  numerical  data 


Further  works  have  to  be  made  on  general  and  coherent  definition  of  the 
notion  of  conflict  (distance)  whatever  the  type  of  data  manipulated 
(numerical,  symbolical)  in  a  general  framework  (Dempster-Shafer) 


O  N  E  R  A 


